Speech applications are designed to respond to spoken human speech and are increasingly being used in a variety of applications. For example, a speech application embodied as an interactive voice response (IVR) application may help users in obtaining answers to their queries, in procuring information related to products and services, in making payments, in lodging complaints, and in general in receiving assistance for a variety of purposes.
The speech applications need to be periodically tested, as the speech applications are prone to errors. However, speech applications pose several testing challenges. For example, comprehensive testing of a speech application requires simulating realistic human speech, including triggering different recognition confidence levels and testing true spoken speech with spoken accents that match expected user usage. Further, many speech applications support multiple human languages, thereby necessitating translation of utterances/prompts, which is often a challenge. In some example scenarios, the speech applications involve multiple iterations thereby requiring changes to grammars and prompts.
In some example scenarios, detecting errors in speech applications may also be difficult, as distinguishing between known and expected behavior requires cross-referencing multiple documents. Further, determining which specific spoken utterances produced specific actions, involves analyzing complex grammars, and as such, determining valid test data is often a time consuming process.
Conventional techniques for manual speech application testing are slow, prone to errors, inefficient at reproducing failures and require substantial human effort to analyze, translate and test the speech applications. Some conventional techniques make use of automated test scripts to increase the speed of testing. However, building test scripts that provide adequate test coverage is a time-consuming and expensive proposition. Moreover, considerable effort is required to regularly update the test scripts as per changes to the speech applications.